Measuring the Use of Factual Information in Test-Taker Essays
نویسندگان
چکیده
We describe a study aimed at measuring the use of factual information in test-taker essays and assessing its effectiveness for predicting essay scores. We found medium correlations with the proposed measures, that remained significant after the effect of essay length was factored out. The correlations did not differ substantionally between a simple, relatively robust measure vs a more sophisticated measure with better construct validity. Implications for development of automated essay scoring systems are discussed.
منابع مشابه
Lexical Chaining for Measuring Discourse Coherence Quality in Test-taker Essays
This paper presents an investigation of lexical chaining (Morris and Hirst, 1991) for measuring discourse coherence quality in test-taker essays. We hypothesize that attributes of lexical chains, as well as interactions between lexical chains and explicit discourse elements, can be harnessed for representing coherence. Our experiments reveal that performance achieved by our new lexical chain fe...
متن کاملArgumentation-Relevant Metaphors in Test-Taker Essays
This article discusses metaphor annotation in a corpus of argumentative essays written by test-takers during a standardized examination for graduate school admission. The quality of argumentation being the focus of the project, we developed a metaphor annotation protocol that targets metaphors that are relevant for the writer’s arguments. The reliability of the protocol is κ=0.58, on a set of 1...
متن کاملScoring Persuasive Essays Using Opinions and their Targets
In this work, we investigate whether the analysis of opinion expressions can help in scoring persuasive essays. For this, we develop systems that predict holistic essay scores based on features extracted from opinion expressions, topical elements, and their combinations. Experiments on test taker essays show that essay scores produced using opinion features are indeed correlated with human scor...
متن کاملDoes number of options in multiple choice tests affect item facility and discrimination? An examination of test-taker preferences
Multiple Choice tests are utilized widely in educational assessment because of their objectivity, ease of scoring, and reliability. This study aimed to compare IF and ID of MC vocabulary test items and attempted to find whether these indices are affected by the number of options. To this end, four 20 item stem equivalent vocabulary tests (3-, 4-, 5-, and 6-option MC) were administered to 194 (1...
متن کاملSpeech-like Pragmatic Markers in Argumentative Essays Written by Iranian EFL Students and Native English Speaking Students
In this study, the use of speech-like pragmatic markers in Iranian EFL students’ academic writing was investigated. Speech-like pragmatic markers, such as I think, well, I guess, actually, anyway, anyhow, etc. are linguistic components that are more specific to conversation than writing, and writers may wrongly include them in their academic writing. To examine the students’ use of speech-like ...
متن کامل